Misconceptions of Roc Analysis

Which eyewitness identification procedure better enables eyewitnesses to discriminate between innocent and guilty suspects? In other words, which procedure better enables eyewitnesses to sort innocent and guilty suspects into their correct categories? The answer to that objective, theory-free question is what policymakers need to know, and it is precisely the information that ROC analysis provides. Wells et al. largely ignore that question and focus instead on whether ROC analysis accurately measures underlying (theoretical) discriminability for lineups. They argue that the apparent discriminability advantage for lineups over showups is an illusion caused by "filler siphoning." Here, we demonstrate that, both objectively and theoretically, the ability of eyewitnesses to discriminate innocent from guilty suspects is higher for lineups compared to showups, just as the ROC data suggest. Intuitions notwithstanding, filler siphoning does not account for the discriminability advantage for lineups. An actual theory of discriminability is needed to explain that interesting phenomenon.